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The discrete-time model of delta modulation is considered for a stationary 
random input process with a rational spectral density, and an auto- 
covariance that goes to zero as the lag approaches infinity. For leaky 
integration, the joint distribution of input and decoded approximation 
processes is shown to approach a unique stationary distribution from any 
initial condition. Under the stationary distribution, the decoded process 
may take on all values in a bounded interval that is independent of the 
input process. For the often-studied ideal integration model of delta modula- 
tion, it is shown that the successive distributions at even parity time instants 
converge to a limiting stationary distribution, while at odd parity time 
instants the distributions converge to a different limiting distribution. 
Under these limiting distributions, the decoded process is assigned a 
positive probability for each level of a (discrete) lattice of amplitudes. The 
mean-absolute approximation error and mean-absolute amplitude of the 
decoded process are shown to be finite under the limiting distributions. For 
both ideal and leaky integration cases, an explicit upper bound on mean- 
absolute approximation error is given, which is independent of the spectral 
density of the input process. 

I. INTRODUCTION 

In spite of the great simplicity of delta modulation as an analog-to- 
digital encoding technique, it has not yet succumbed to an adequate 
mathematical analysis. Although realistic inputs such as speech are 
extremely difficult to characterize, considerable insight could be ob- 
tained from a thorough analysis for the case of a stationary random 
input process with a prescribed spectral density. Yet no such results 
have been obtained because of the mathematical complexity of the 
nonlinear feedback loop. In fact, the presence of a feedback loop raises 
the possibility that instability in some sense could arise. The possibility 
that the decoded signal could "run away" or become unbounded, 
failing to track the original signal, has never been theoretically excluded. 
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Although experience with delta modulation shows that such an extreme 
form of instability never arises, it has never been shown analytically 
that the mean-square or mean-absolute quantizing noise has a finite 
upper bound. Another possibility which has never been theoretically 
excluded is erratic operation, where the statistical average of the 
quantizing noise magnitude continues to vary with time. In other 
words, although the input process is stationary, the decoded approxima- 
tion process would be nonstationary with a time-varying probability 
distribution even after low-pass filtering. If this were the case, the 
decoded process would not be replicating the original process very 
effectively. 

Recently, D. Slepian 1 has developed an exact computational approach 
for finding the joint probability distribution of the original and encoded 
processes. These results make it possible to accurately compute such 
curves as the mean-square quantizing noise versus step size for particular 
spectral densities of the input process. Slepian's results are based on 
the initial assumption that, for a stationary input process with rational 
spectral density, the joint probability distribution will approach a 
unique stationary distribution from any starting condition. (For delta 
modulation with ideal integration, the stationary distribution actually 
refers to half the sum of the distributions at two successive time instants 
to account for the well-known parity change between even and odd 
amplitude levels.) On a practical level, this stationarity assumption 
seems to be entirely reasonable; yet it has never been theoretically 
justified. 

Other authors have also assumed stationarity. In particular, for ideal 
integration H. van de Weg 2 assumed implicitly that the decoded process 
had two different stationary distributions for the even and odd parity 
time instants. D. J. Goodman 3 assumed a random phase initial condition 
so that only one stationary distribution, half the sum of the even and 
odd parity distributions, need be considered. In both cases the assump- 
tion is implicitly made that for any initial condition the delta modulation 
process will approach a steady-state mode of operation with a separate 
stationary distribution for even and odd time instants. 

As a final argument to point out the need for an analysis of stochastic 
stability properties of delta modulation, consider the fact that most 
heuristic and semi analytical approximate considerations of delta 
modulation are based on the model of an ideal integrator in the feedback 
path, while most physical realizations involve a leaky integrator. There 
is a basic qualitative difference between these two cases, even for ex- 
tremely wide-band integrators. This is because ideal integration gives 
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equal weight to a current input sample and an arbitrarily remote past 
input sample, while leaky integration forgets remote past samples. 
Hence, it is not clear that the ideal integration model is meaningful, 
even if it is known that the leaky integration system is well behaved. 
To justify the validity of using an ideal integration model, a rigorous 
demonstration of the stability of this model is needed. 

This paper demonstrates the stochastic stability of delta modulation 
for both ideal and leaky integration by giving a mathematical proof 
that the joint distribution of the input and decoded processes approaches 
a unique stationary distribution from an arbitrary starting point and 
by deriving an explicit, finite upper bound on the mean-absolute quan- 
tizing noise. The input process is assumed to be stationary and con- 
tinuously distributed, with finite variance, a rational spectral density, 
and an autocovariance that approaches zero as the lag goes to infinity. 
A shaping filter with white noise input is assumed as the generating 
mechanism of the input process. 



II. PROBLEM FORMULATION AND SUMMARY OF RESULTS 

For most purposes the following discrete-time model of delta modula- 
tion is an acceptable description of the actual continuous-time opera- 
tion. Let u, denote the sampled analog values of the input process at 
successive time instants t = 0, 1, 2, • ■ • . The time scale is normalized 
for convenience without loss of generality. The delta modulator shown 
in Fig. 1 generates a binary-valued process b k according to 



b k = sgn (u k - x k ) 



(1) 



where sgn y = +1 if y ^ and — 1 if y < 0, and x k is the decoded 
•process which approximates u k . The decoded process x k is given re- 
cursively by 



x h+i = x k + A sgn (u k - x k ) 



(2) 
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Fig. 1 — Delta modulator. 
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for the ideal integration case, and by 

x k+1 = ax k + A sgn (u k — x k ) (3) 

for the leaky integration case with a simple RC integrator, where A > 
is the step size and \ ^ a < 1. (In practice, a is very close to one because 
the integrator time constant is much longer than the sampling period.) 
The quantizing noise, 

e k = u k — x k , 

is the error at time k due to the analog-to-digital-to-analog processing 
of the delta modulation system. 

Assume the input process u k is stationary, continuously distributed 
with finite variance, and has autocovariance approaching zero as the 
time lag goes to infinity. For convenience, assume also that the prob- 
ability density of u k is everywhere positive so that, as in the Gaussian 
case, there is a positive probability of u k lying in any open interval. 
Note that u k is not assumed to have zero mean value. 

A summary of the results obtained in this paper follows. Let F k 
denote the joint distribution of the input and decoded processes u k and 
x k at time k. 

2.1 Ideal Integration 

(i) For any initial condition of the form x { = wA + with m an 
integer and | 6 | ^ A/2, the two sequences of distributions [F 2k } and 
{^2*+i} separately converge to unique stationary distributions G n 
and (?, , respectively. One distribution assigns positive probability for 
the process x k — 6 to even integer multiples of A, the other distribution 
to odd integer multiples, depending on the even-odd parity of the 
initial integer ???. 

(ii) With these stationary distributions the mean-absolute quantizing 
noise, averaged over two successive time instants, is bounded according 
to 

\[E | x k - u k | + E | x k+l - u k+1 |]£*|«,| + A/2 + 2|#|. (4) 

2.2 Leaky Integration 

(i) For any initial value of Xi , the distributions \F k ] converge to 
the unique stationary distribution (r, under which x k may take on all 
values in the range 

I * ! < ~ (5) 

1 — a 
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(ii) Under this stationary distribution, the mean-absolute quantizing 
noise is bounded according to: 

E | X k - U k | ^ E | M t | + A/2a. (6) 

Note the qualitative difference between the leaky and ideal integration 
cases. In the leaky case, r k is distributed over a finite interval; in the 
ideal case, x k is discretely distributed on a lattice. 

III. MARKOVIAN MODELING 

Since the input process u k is stationary with finite variance and 
rational spectrum, then u k = u k — y. (where n is the mean value of u k ) 
can be modeled 5 as the response of a stable discrete-time shaping filter 
to a zero mean, finite variance "white noise" process iv k (with iu k inde- 
pendent of iv k -i for i = 1, 2, 3, ■ ■ ■ ). More precisely, a white process 
w k and a stable rational shaping filter H(z) can always be specified in 
such a way that the response u k of the filter to the excitation w k will 
be a stationary random process with spectral density identical to that 
of u k . If w k is also chosen to be continuously distributed with a positive 
density, then# A will also satisfy this property. Thus, all the assumptions 
made in Section II about the process u k are possessed by the process 
il k + /*. It is therefore reasonable to study the effect of the delta modula- 
tion system for the input u k + n, whose structure or generating mecha- 
nism is known. For the remainder of this paper, no distinction will be 
made between u k and u k . 

Using this model and the assumption that the autocovariance of u k 
goes to zero as the lag approaches infinity, Appendix A shows that 
u k can be imbedded in a vector Markov process, d* , with 

d* = (d ki , d k2 , • • • , d kn )' 

and 

M* = d kl + n (7) 

where n denotes the dc value of the input process and n is the number 
of poles in the shaping filter. The vector d k characterizes the state of 
the filter at time k and is generated by the recursion 

d* + 1 = Ad* +bw* (8) 

where A is an n X n matrix with eigenvalues within the unit circle, 
and b is a fixed vector. The process d k is Markovian, since the condi- 
tional distribution of d*+i given all past states d k , d t _, , • • • , depends 
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only on the given value of d* . Appendix B shows that for any initial 
state d , the distribution of d* approaches a unique stationary distri- 
bution. Equations (2), (7), and (8) for the ideal integrator case, or 
eqs. (3), (7), and (8) for the leaky integrator case, jointly characterize 
the evolution in time of a Markovian process whose state s* at time k 
is given by the n + 1 component vector 

s* = (x k , u k , d k2 , d k3 , • ■ • , d kn )'. 

Then, given the value of s k , the distribution of a k+l is completely 
determined. Henceforth, a distribution F k , describing the joint distribu- 
tion of the n + 1 components of the vector s* , will be regarded as a set 
function which assigns a probability F k {A) to the region A of the n + 1 
dimensional space of possible values of the state vector s k . The prob- 
ability transition function 6 characterizing the Markov process is defined 
as 

p(s, A) = P{s k+l zA | s k = s) 

which is independent of k. By averaging this conditional probability 
over a distribution F k assigned to s* , the unconditioned distribution 
F k+l of s t+ i is obtained: 

F k+l (A) = J p(s, A)F k (ds) = E FkV (s, A). (9) 

Thus, F k+i is related to F k by a linear mapping T, so that in operator 
notation 

F k+1 = TF k . (10) 

Note that T plays the same role as the probability transition matrix 
in Markov chains. The process has a stationary distribution G, if G = TG, 
so that G is self-reproducing. If any state vector has distribution G, all 
subsequent state vectors will have this distribution. The existence of 
a stationary distribution is a necessary but not sufficient condition 
for the convergence of the distributions ^. +1 to a limiting distribution. 
The Markovian model will be used in Sections V and VI to obtain 
the convergence properties of the distributions {F k }. But first, it is 
necessary to obtain a bound on the time and ensemble average of the 
quantizing noise. 

IV. BOUNDING THE TIME-AVERAGED MEAN-ABSOLUTE QUANTIZING NOISE 

Suppose that at the initial time instant k = 1, the system has an 
arbitrary initial state s, . Appendix B shows that for any initial state 
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d! of the shaping filter, the process d t , and hence u k , converge in 
distribution and E \u k \ converges to c, the mean-absolute value under 
the stationary distribution of the process u, . 
Squaring both sides of eq. (3) gives 

a£ +] = axl — 2a A | u k - x k | + 2a Am* sgn (u k — x k ) + A 2 

g xl - 2a A | U k - a:* I + 2a A | u k \ + A 2 . 

Taking the expected value of both sides yields 

E4 +i ^ Ell ~ 2a AE | e k | + 2aAE \ u k | + A 2 

and iterating backwards gives 

Ex* k+1 ^ xl + 2aAJKE\u i \-E\ e< |) + A 2 /c. 

i = l 

Since the left side is nonnegative, it follows that 

zT,E\e t \£ x]/2aAk + \ J} % I «< I + A/2«. (11) 

Hence, using the fact that E \ u, \ — > c, 

1 * 

lim sup j E E I e > : I ^ c + A /2«, (12) 

which shows that the long-term time average of the mean-absolute 
quantizing error is bounded. Since the preceding derivation holds for 
a ^ 1, it applies for both leaky and ideal integration, setting a = 1 
for ideal integration. 

For leaky integration, the decoded process is, in fact, bounded 
deterministically. Integrating eq. (3) backwards yields 



so that 



Hence, 



x k+l =«\r, + A £,«*-% (13) 



**, I * «* i *i I + 7^- d - «*)• d4) 

1 — a 



lim sup I Xk I ^ i (15) 

Thus, the decoded process is bounded with probability one in the case 
of leaky integration. 
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V. EXISTENCE OF A STATIONARY DISTRIBUTION 

A sequence of random vectors and the corresponding sequence of 
distributions are said to be stochastically bounded if, for any probability e, 
however small, there is a sufficiently large distance R such that each 
random vector of the sequence has probability less than e of having 
length greater than R. Hence, the successive vectors cannot have a 
positive probability of moving out toward infinity. 

For a sequence of distributions {F k \, define the associated sequence 
of averaged distribution \G k \ by 

G k (A) = lt,/<(A). (16) 

Thus, if / is a randomly selected time instant from the first k integers 
each having equal probability, then G k (A) is the probability that the 
random vector s r lies in a region A, where s, has distribution F, . 
If the sequence {F,-} is stochastically bounded, then the averaged 
distributions {G k } are also stochastically bounded; however, the converse 
is not always true. 

To show that the Markov process s* defined in Section III has a 
stationary distribution G, the following theorem, proved in Appendix C, 
may be used. 

Theorem: A Markov process has a stationary distribution if 

(i) for any initial state Si , the averaged distributions G k are sto- 
chastically bounded, and 

(u) for any region A, let D be the set of points s at which the transi- 
tion probability function p(s, 4) is discontinuous and let N s be 
the set of all points whose distance from D is less than 8; then 
there is a function C(5) independent of s with C(5) — > as 5 — > 
and for all s, 

p(s, N t ) ^ C(8). (17) 

Condition (i) excludes the possibility that successive state vectors 
can move out toward infinity. Condition (it) is concerned with the 
region of state space where an arbitrarily small perturbation of a given 
state vector can cause a substantial change in the induced distribution 
of the state vector at the next time instant. This region is contained 
within the set N 6 for each 5 > 0. The condition requires that this region 
be suitably unimportant. 

To show that the delta modulation process satisfies condition (i), 
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observe that 

E | Xt | ^ E | Ui - Xi | + E | ut | 

and since Z£ | m,- | is bounded, eq. (11) shows that for some constant K, 

! t^K- | <K. (18) 

Now Chebyshev's inequality, 

MP {| a;,- | > M} £E\ Xi | , 
applied to eq. (18) yields 

7 EP{|as« | > M| ^ K/M 
ft ,=i 

for every Af > 0, which shows that the averaged distributions of the 
decoded process x k are stochastically bounded. But from Appendix B, 
the d k process is stochastically bounded. Hence, the averaged distribu- 
tions of d k are also stochastically bounded. Thus, the marginal distribu- 
tions of the joint distributions G n are stochastically bounded so that 
G n is itself a stochastically bounded sequence, and condition (i) holds 
for the ideal integration case. For the leaky integration case, condition (i) 
is satisfied since eq. (15) shows that the x components of the vectors s* 
are uniformly bounded with probability one, so that the above argument 
shows that the joint distributions G„ are stochastically bounded. 

To verify condition (it), note from eqs. (2) or (3) that for any region 
A, p(s, A) is continuous, except in the set D of all points s the first two 
components of which, x and u, are equal. Appendix D shows that given 
s* , the variate u k + 1 is continuously distributed and that this implies that 
there' exists a function C(5) which goes to zero as 5 approaches zero and 

P\\x-u k \\s k } ^ C(8) (19) 

where C(8) is independent of x and s k . But since x k+l is completely 
determined by s t , (19) implies that 

P{\x k+1 - u t+] | < 5|s*} ^ C(5). (20) 

Hence, eq. (17) is satisfied and condition (it) holds for both ideal and 
leaky integration. Therefore, a stationary distribution exists. 

VI. ALLOWABLE AMPLITUDE VALUES FOR THE DECODED PROCESS 

For ideal integration, it is clear from eq. (2) that an initial condition 
of the form 
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x v = mA + 

with m an integer and | 6 \ < A, implies that all subsequent amplitude 
values of the decoded process will be confined to the lattice 

x k = ia + e l = 0, ±1, ±2, ••• . 

Since the preceding results did not specify a particular choice of initial 
condition, it follows that for each 0, a stationary distribution G„ exists. 
For convenience, assume = 0. No loss of generality will result because, 
for any 0, the problem can be converted to the = case by replacing 
the input process u k by u k — as can be seen from eq. (2). This simply 
changes the dc value of the input by, at most, one step size A. 

Under the assumption that u k has a positive density, it follows 
from eq. (2) that there is always a positive probability of either increas- 
ing or decreasing by A in going from x k to x k+1 . This means that every 
integer multiple of A must have a positive probability under the station- 
ary distribution, because each level can always be reached from any 
other level in a finite number of steps. (On the other hand, if u k were 
bounded, then eq. (2) shows that the decoded process would get locked 
into a bounded set of levels from which it would never escape.) 

For leaky integration, the situation is strikingly different. The 
decoded process will get locked into the bounded region 

X = {x: -A/(l - a) £ x g A/(l - «)}. (21) 

This may be seen by noting from eq. (3) that if x k is in X, then x k+ i 
must also be in X. Consequently, once x, is in X, it will never escape. 
If x k is not in X, then eq. (3) shows that 

I x k+ i [ <C | Xt | 

and if b k has the correct polarity, 

| x k+1 | < a | X k | . (22) 

Hence the values | x k+i \ must decrease monotonically as long as x k+i 
remains outside of X. Since u k has a positive density, 6, must have 
positive probability of having either polarity; which means the stronger 
inequality eq. (22) must hold at some subsequent time instants. Hence, 
the process must eventually enter the region X. 
Iterating eq. (3) backwards yields 

x k+1 =a k x i + A X>'V. • (23) 

1 = 

This shows that the initial value x v is gradually forgotten and the set 
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of allowable values for x k as k goes to infinity, approaches the set 

W = [x\ X - A(±l ±a ±a ± • • • )) (24) 

where all possible sequences of polarities are used to generate values 
of x. Appendix E proves that, for a ^ \ , the set W coincides with the 
interval X* 

For the leaky integration case, the stationary distribution G clearly 
must confine the x component of the state vector to the region X. 
Furthermore, the following argument shows that G assigns a positive 
probability to every open subinterval (y — e, y + e) with y in X and 
e > 0. Let [c t ] be a suitable binary sequence (generated as in Appendix E) 
satisfying 

1/ = aEc«''. (25) 

1=0 

Pick the integer N large enough so that 

a N+1 Xl < e/3, and a" A/(l - a) < e/3, (26) 

and consider the event 

E = {62 = Cjv_i , 63 = C?/-2 , • • • , Ojv + i = Co), 

which has a positive probability for any initial state s, . Then for 
k = N -\- 1, eq. (23) can be written in the form 

00 

X N +2 - V = <X N+1 Xt + Ac/fc, - A ^a'Ci 

t=A r 

so that, using eq. (26), 

I r N+2 - y | g «/3 + «/3 + e/3 = e, 

for Ji in X. Hence, 

P[\x H+a - y\ < e|s,l ^ Pf^ls,} (27) 

and averaging eq. (27) over the distribution G for the state Si shows that 

P\\x N+2 - y\ < e} > (28) 

where 3-^+2 has the marginal distribution of the first component of G. 

VII. CONVERGENCE TO THE LIMITING DISTRIBUTIONS 

The following specialized and paraphrased version of a theorem due 
to J. L. Doob 7 is suited to the delta modulation process: 



* For a < \, it can be shown that W does not coincide with X, in fact W is not 
even dense in X. 
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Doob's Theorem: Suppose the Markov process has a stationary distri- 
bution Q and satisfies the conditions: 

(i) For any initial state s t , if A is a region with G(A) = 0, then 

F k (A) -* 0. 
(ii) If a region A satisfies p(s, A) = 1 for s in A, then G(A) = 1. 

Then either 

F k —* for any initial state 

in which case the process is aperiodic, or there exist m disjoint sets 
A , A x , • ■ • , A m -i with G(\J m A { ) = 1 and p(s, A i + i ) = 1 if s is in A t 
for i = 0, 1, • • • , m — 2 and if s is in A m - V , then p(s, A ) = 1. In 
this case, the process is said to be periodic with period m. In particular 
for m = 2, there exist two distributions G and G'i with G = \(G Q + G t ), 
G t (Ai) = 1, GoiAn) = 1, and for any initial state in Ai , F 2k+1 — > Gi 
and F 2k — * G , while for any initial state in A , F 2 * + i — ► G and F 2k —* G Y . 

For ideal integration, Section VI shows that there are no transient 
levels for the x k process and Appendix A shows that d k has a positive 
probability of lying in any region of n space with nonzero volume. 
Hence, there is no transient set A with G(A) = except for trivial 
sets with F k (A) = for each k. For leaky integration the only nontrivial 
sets A are regions where the x component lies outside of X and for 
such regions, Section VI shows that F k (A) — > 0. Therefore, condition (i) 
is satisfied for both types of integration. 

Furthermore, the ergodicity requirement (ii) is also seen to be satisfied 
for both ideal and leaky integration from the results of Section VI. 

For ideal integration the process clearly has period 2, since, if A is 
the set of all state vectors with x components taking on even integer 
multiples of A, and A x is the complementary set, then A \J A x has 
probability one under the limiting distribution and the transition 
probability function has the requisite property implying the state 
vector alternates between A and A t . For leaky integration the process 
is aperiodic since eq. (27) holds for all N sufficiently large so the process 
cannot satisfy the requirements for periodicity, hence F k —* G. Since 
a sequence of distributions cannot at the same time converge to two 
different distributions, it follows that the stationary distribution G is 
unique for both ideal and leaky integration. 

VIII. BOUNDING STEADY-STATE QUANTIZING NOISE 

The fact that a sequence of distributions converges to a limiting 
distribution does not imply that moment functions such as mean or 
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variance converge to the corresponding moment of the limiting distri- 
bution. However, it does imply that the sequence of expectations of a 
bounded continuous function converge to the expectation of that func- 
tion under the limiting distribution. It turns out that this property 
is sufficient to obtain a bound on the least mean-absolute quantizing 
error under the stationary distributions. 

For ideal integration, eq. (12) can be rewritten in the form 

lim sup — £ E(\ e» | + | e 2k+1 |) ^ K (29) 

where K = c + A/2, which implies the existence of an even subsequence 
of time instants t t (= 2/c.) with 

|£(| e tt | + | e li+1 |) ^ K. (30) 

Now define the truncating function J H (e) according to 

1 | e | ^ R 

j R (e) « Ji - (| e | - R)/s R < \ e \ < R + 5. 
R + 6 ^ \e | 

Then, 
#(| *n I + | a„ + i I) ^ #{| e<. I -MO + I «n+i ! /«(e«« + i)) (31) 

and, since the right-hand side is the expectation of a bounded continuous 
function, 

E[\e u | •/«(«,,.) + |e, 1+ i | /*(e, (+ i)} -» #o I e, | J R (e,) 

+ E x | e,- +1 | J*(e, + 1 ) (32) 

where ; denotes an even time instant in steady-state operation and 
E„ and E x denote the expectation under the distributions G and G x 
respectively, or reversed, depending on the parity of x v . Since eq. (32) 
holds for each positive R and 5, taking the limit as 5 — * and R — > °o 
shows that 

\{E n I e.- | + E x | e. +1 |) ^ c + A/2. (33) 

Thus, the mean-absolute quantizing noise averaged over two consecutive 
time instants has the bound c + A/2 under the limiting distributions 
for ideal integration. 

For leaky integration, the process T k is bounded with probability one, 
so that in this case all moments of r k converge to the corresponding 
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moment under the limiting distribution. Furthermore, in Appendix B, 
it was shown that the first absolute moment of u k converges to the 
corresponding value under the limiting distribution. Together, this 
implies that E \ e, ■ | converges to E a \ e, \ . Hence, eq. (12) yields 

Eo | e,- | ^ c + A/2a. (34) 

This result could also have been obtained by the same argument used 
to derive eq. (33). 

Note that the bounds, eqs. (33) and (34), are independent of the 
spectral density of the input process u k and are therefore very crude 
bounds. An important feature is that the mean-absolute quantizing 
noise is shown to be finite under the stationary distributions. An im- 
mediate consequence is that the decoded process x k has a finite first 
absolute moment for ideal as well as leaky integration. Since 

E | Xi | = E | e, + m, | ^ E | e t | + E | w, | , 

then 

E G | z< | £ 2c + A/2a (35) 

for both leaky and ideal integration. (Set a = 1 for ideal integration.) 
Possibly of interest also is that this bound may be used to obtain upper 
bounds on the tail probabilities of the decoded process by using the 
Chebyshev inequality. 

As discussed in Section VI, ideal integration with initial values of z, 
of the form mA + can be handled by replacing u k by u k — 6, so that 
the bound, eq. (35), remains valid if c is replaced by c + | \ , and 
| e, | by | e, | — | 6 | , which leads to the inequality, eq. (4). 

IX. CONCLUSIONS 

The results of this paper show that delta modulation indeed possesses 
the qualitative properties of convergence to a stationary distribution 
and boundedness of the quantizing noise. Perhaps of greatest interest 
is the fact that the results also hold for ideal integration, thus justifying 
the study of this idealized model to obtain an understanding of the 
usual physical situation of leaky integration. 

The use of a Markovian model of the input process has been con- 
sidered by several authors 8-11 as an approach to determine actual 
probability distributions for the steady state. The results of the paper 
show that the Markovian recursion, i.e., the usual Chapman-Kolmogorov 
equation, will, in fact, converge to the unique stationary distribution. 
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The technique used here for showing the existence of a stationary 
distribution (an invariant solution to the Chapman-Kolmogorov 
equation) extends the method used by this writer 12 for an adaptive 
filtering algorithm and the earlier results for Feller processes. 13 ' 14 



APPENDIX A 

Markovian Imbedding 

Suppose initially that the process u k has zero mean. Then u k is 
generated by the recursion 

U k+n = a i u k+n - 1 + • • ■ + a n u k + 0iiv k+n -! + • • • + P„w k (36) 

with /3 7^ 0. This equation describes the operation of a stable shaping 
filter with transfer function 



-» ■ g 



i=i 

n V^ n-i 

I — 2-t a i Z 



with A(z) having all roots inside the unit circle, \z\ < 1, and w k is a 
white process with zero mean, finite variance and an everywhere-positive 
probability density function. The requirement that the autocovariance 
goes to zero for lags approaching infinity is satisfied by the fact that 
B(z) is of lower degree than A(z). 
The state vector d* is defined by 



d lk = u k - m 
d i+ i, k = dt, k+l — biW k i = 1, 2, • • • , n — 1 



(37) 



which when combined with eq. (36) leads to the state equations 

d* +1 = Ad k + bw t (38) 

characterizing a vector Markov process. The matrix A is given by 

10 ••• 

1 ••• 
• 



A = 







... 1 

• • • Oi 
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and the constant vector b = (6, , b 2 , ■ ■ ■ , &„)' is determined by solving 
the equations: 

&, = A 

. = 0, + a,-_,&i + • • • + a 2 6,_ 2 + ai&i-i for i = 2, 3, • • • , n. 

The matrix A is stable since its eigenvalues are the roots of A (z). This 
state representation is a standard one in the control literature. See 
for example, Ref. 15, p. 221. 

The polynomials B{z) and A(z) may be assumed to have no common 
roots so that the shaping filter is intrinsically of order n. Then, the 
state generating eq. (38) is known to be completely controllable.* This 
means that, for any initial condition at time k, it is always possible 
to find values for w k , w k+l , • • • , w* + —i to produce any desired value 
of the state vector d k+n . It follows that since w h has a positive density, 
the state vectors d k have a positive probability of lying in any region 
of n-space with nonzero volume. 

APPENDIX B 

Convergence in Distribution of d k 
The Markov process defined by eq. (38) can be iterated to obtain 



k 



d k+1 - A" d x + 2 A'-'bwt . (39) 

Since A has all eigenvalues of less than unit modulus, A k — > as k — > « , 
so that the first term on the right side of eq. (39) goes to zero with 
probability one. The second term has the same distribution as 



Vt 



= ^ A'biVi 



since the variates iv k are independent and identically distributed. But 
v k is a martingale, 16 since 

E\v k+i | v k \ - v t 

and 

*-i II & II 

E 1 1 v k 1 1 ^ Y, y 1 1 & 1 1 E I Wi I < ^ — 4" -& I w» 

f-o 1 — A 

is finite, where X denotes the Euclidean norm of A, X < 1. Then by the 
* See Theorem 7-8, p. 389 of Ref. 15. 
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martingale convergence theorem, v k converges with probability one to a 
random variable v n . Hence, the probability distribution of the vector 
dfc+i converges to the distribution of v^ , where 

00 

v„ = £ A'"6u>< . (40) 

i = 

Since w k is uncorrected and has finite variance, it may be seen from 
eq. (40) that v M also has finite variance. This, together with the con- 
vergence in distribution, implies (Ref. 6, p. 252) that the mean absolute 
value of each component of d* converges to the mean absolute value 
of the corresponding component of v«, . Consequently, 

E | u k | -> c 

for any initial state di , where c is the mean absolute value of the first 
component of v n . 

appendix c 

Existence of a Stationary Distribution 

Theorem: A Markov process has a stationary distribution if 

(i) for any initial state s t the averaged distributions G k are stochastically 

bounded, and 
(ii) for any region A, let D be the set of points s at which the transition 
probability function p(s, A) is discontinuous and let JV S be the set 
of all points whose distance from D is less than 5; then there is a 
function C(8) independent of s with C(8) — > as 5 — > and for all s, 

P(s, N s ) ^ C(8). 

Proof: Since the sequence of averaged distributions G k is stochastically 
bounded, by the Helly selection theorem, Ref. 6, p. 267, there exists a 
subsequence G ki converging to a limiting distribution G. From the 
definition of G k and T it follows that 



so that 



TG n = G n + - (F n+l - F t ) 



TG ni (A) - G ni (A) I < — -> 
1 — n,- 



as i — » oo for any region A. Since G„. converges to G, it then follows that 
TG ni ->G i-^oo. (41) 
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It remains to show that 

TG ni -> TG (42) 

so that eqs. (41) and (42) will imply that G = TG, which is the desired 
result. 

To prove (42), note that (?„, —> G implies that for any bounded 
continuous function <p of the state vector (s), 

E i<P (s) -> Eg<p(s) (43) 

where E t is the expectation under (?„, . If p(s, A) were continuous 
in s, eq. (42) would follow from eq. (43) by setting <p(s) = p(s, A) 
and noting from eq. (9) that 

E iP (s, A) = TG ni {A). 

However, p(s, A) is not itself continuous and the following argument 
is needed to complete the proof. 

Let J«(s) denote the function which is equal to 1 if s is not in N t , 
and for s in iV 8 let I«(s) denote the distance of s from D. Then 
I s (s)p(s, A) is bounded and continuous in s for any region A, and so 

#,.{/a(s)p(s, A)) ->E G {U{s)p{s,A)), »-> oo. 
But 

E iP (s, A) - ^{/,(s)p(s, A)} ^ G ki (N s ) (44) 

and also 

E a p(s, A) - E {I s (s)p(s, A)} ^ G(N S ). (45) 

But since 

p(s, N s ) ^ C(8) 

by hypothesis, it follows by averaging over s that 

F h (N,) ^ C(8) 
and therefore 

G ki (N 6 ) ^ C(8) (46) 

and, since G ki (N 6 ) — > G(N S ), then 

G(N S ) ^ C(8). (47) 

Combining these results shows that 

limsup | E f p(s, A) - E G p(s, A) | < 2C(8), 
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but since 8 can be made arbitrarily small, it follows that 

E iP (s, A) -> E G p(s, A). 
Hence, eq. (42) holds and the theorem is proved. 

APPENDIX D 

Existence of C(8) 

From eq. (38), it follows that 

u 4+ i = (Ad k )i -f biW k 

with 6, ?* 0, so that the conditional distribution of u k+1 given s* is 
continuously distributed because w k is continuously distributed. Let 

I = (Ad k )i , and let 

H(x) = P{b i w k < x). 

Then 

P{u k+1 < x | s k \ = H(x - I) (48) 

which is a uniformly continuous function of x. Therefore, if 

C(B) = sup [H(x + 5) - H(x - 8)] 

then 

C(8) -> as 8 -> 0. 
But 

P{\ u k - x I < 5 | s*} = H(x + I + 5) - H(x + I - 8) g C(«) 

using eq. (48), which proves the existence of a suitable function C(8) 
independent of s t and x. 

APPENDIX E 

Range of the Mapping y = ±1 ± a ± a 2 ± •■• 

Theorem: For f ^ a < 1, the range of values taken on by the mapping 

CO 

y - S «•&,- (49) 

t=0 

for all binary sequences, b, : = ±1 eac/* t, t's Me closed interval \y\ ^ 1/(1 — a) . 
Proof: For each ?/ in the interval \y \ ^ a, with a = 1/(1 — a), generate 
a binary sequence b , bi , b 2 , ■ ■ • according to the algorithm below. 
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Let 

V = £(2/ + «)• 
If p ^ 1, let /o = 1 otherwise /„ = 0. Let 

s„ = Jb«/< n = 0,1,2, ••• . 

For n = 1, 2, 3, ■ • • , if p - s n ^ <x n+1 let / n+1 = 1, otherwise / n+1 = 0. 
Then 

bi = 2/,- - 1, i = 0, 1, 2, ■ • • . (50) 

To prove that eq. (49) holds for the binary sequence generated in this 
manner, note first of all that for all n ^ 0, 

s„ ^ s n+1 , and s n ^ p 

so that s n -> s for some number s with s ^ p. Suppose that s < p. 
Then there exists a largest integer m satisfying 

p < s m _! + <x m . (51) 

Therefore, / m = 0, and / m+ , = 1 for each i > 0. Consequently, 

■ m + l I m + 2 I 

p > S = s m _i + a -\- a -f- ' ' ' 
so that 

p > Sm _x + « m+1 /(l - «)• (52) 

But eqs. (51) and (52) imply that 

m+l 

1 — a 
so that 

« < h 
which is a contradiction. Therefore, 



V 



= E«7. 



■ =0 

and so 

V - S «*'(2/.- - a) = £ a'b. , 

,=o »-o 

which proves the theorem. 
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